Data storage systems may employ hardware bus systems to provide fast data transfer from hosts such as network servers via the bus system to attached data storage servers having storage devices, cache storage, or nonvolatile cache storage. It is advantageous to provide data storage that operates at relatively fast speeds which approach or match the speeds of the host processors, or which release the host processors during write operations, such that the host processors are not slowed.
The data to be stored through a write command sent across a hardware bus is typically customer data which will be retrieved at a subsequent time. It is of the utmost importance to the customer that the customer data not be lost or compromised. Thus, in addition to fast data storage, there must be some assurance that customer data that has been transmitted across a bus system was received intact. Hence, most channel adapters for the hosts require an acknowledgment that a data transfer write operation has completed successfully.
One type of hardware bus commonly employed in a data storage system is the Peripheral Component Interconnect (PCI) bus system. A PCI bus system is a high performance expansion bus architecture which offers a low latency path employing PCI bridges through which a host processor may directly access PCI devices. In a multiple host environment, a PCI bus system may include such functions as data buffering and PCI central functions such as arbitration over usage of the bus system.
In PCI and other bus systems, the channel adapters perform write commands to transfer data to their destinations. Channel adapters may poll (a read command) a hardware indicator that signals “end of transfer,” or issue a read command to a location as far down the data path as possible in an attempt to acquire an acknowledgment that the data transfer has completed successfully.
Read operations across a hardware bus are slow and inefficient because a substantial wait is required while the written data is accessed and loaded for passage back through the bus system. Therefore, bus architecture typically blocks the read request from the bus until after the requested data is loaded so as to allow other uses of the bus by other requesters during the load process. An element of the arbitration typically employed by a bus is that the adapter sending a read command must receive a response within a predetermined time or the requester will have to give up the interface as the arbitrator cycles to the next agent having work. As a result, the originator continues to request the read results which are ultimately provided to a buffer to match the request. Therefore, read commands across a bus are extremely slow operations to complete, especially in complex bus systems with multiple hosts and with multiple bridges between the channel adapters and the storage system. During the time required to complete the read, the host adapter that originated the writes which are being verified must pause and wait for the read command to be completed before receiving an acknowledgment that the write operation completed successfully, effectively locking up the host adapter.
The problems attendant with extremely slow read operations across a bus are in part addressed by the use of write command verification as described in commonly assigned U.S. Pat. No. 6,535,937, entitled WRITE COMMAND VERIFICATION ACROSS A PCI BUS SYSTEM, issued Mar. 18, 2003, which patent is incorporated herein by reference in its entirety. The method and system described in U.S. Pat. No. 6,535,937 provides for the asynchronous and prompt verification of a write command across a bus, in particular a PCI bus system. U.S. Pat. No. 6,535,937 discloses a system where the target of the writes has a predetermined “echo” address to which it will respond with an echo write to indicate that the data has arrived at its destination. Thus, the number of reads necessary to verify the successful transmission of customer data is minimized. However, prior systems such as that described in U.S. Pat. No. 6,535,937 do not indicate how many transactions have been processed in a data streaming environment. Thus, prior systems write useful information to the echo location, but provide no way for determining how much data has reached its destination without employing multiple poll or read commands.